1. Field of the Invention
The present invention relates to processing image data in response to data structure defining a sequence of image processing, and in particular relates to processing image data at multiple frame rates.
2. Description of the Related Art
Systems for the processing of image data have historically required multiple processing steps in order to combine several clips into a single composited scene. Examples of this technique are where several moving images, for example models of spaceships, are superimposed upon a background that comprises actors, and maybe several items of computer-generated artificial scenery.
More recently, it has become possible to composite scenes of considerable complexity by defining complex sequences of effects, and other types of image processing in a data structure called a process tree. This data structure is interpreted by a computer to render an output clip on a frame-by-frame basis, rather than by generating multiple intermediate clips.
A process tree typically has several inputs, and an output. A frame supplied at an input is processed according to process nodes connected between the input and the output. A process node may combine frames from a plurality of sources. A process tree may contain hundreds of such processing nodes, and thereby achieve remarkable realism when compositing scenes of an artificial nature.
When intermediate clips have to be generated, this prevents the results of processing from being previewed interactively. This way of working is extremely time consuming and expensive. The process tree takes from less than a second to several minutes to render, depending on the complexity of interconnected processes, but nevertheless the results of changes made in the processing tree can be seen relatively quickly, thereby making this an extremely productive method of working.
A process tree is interpreted by the image processing system to define a processing pipeline, through which images propagate a frame at a time, in order to generate each requested output frame. However, this frame-by-frame approach restricts the connected processes to those that have an input frame rate that precisely matches the output frame rate. Standards conversion processing, and timewarp processing must be performed using conventional techniques, using intermediate clips. This limitation reduces the value of the process tree method, particularly in image processing environments where there is an increasing requirement to perform image processing upon mixed format image data.